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Abstract 

The  DoD’s  evolutionary  acquisition  policy  is  directed  against  project  risk,  but  bears 
inherent  risks  of  its  own.  The  DoD  policy  for  evolutionary  acquisition  mandates  multiple  product 
releases  via  spiral  (i.e.,  amorphous  &  unplanned)  or  incremental  (i.e.,  defined  &  deferred) 
development  methodologies  for  all  programs.  All  amorphous  spirals  eventually  become 
definitive  increments.  Incremental  development  entails  the  deliberate  deferral  of  work  to  a 
subsequent  phase.  Computational  organizational  modeling  using  systems  dynamics  reveals 
that  this  methodology  introduces  more  concurrency  during  development,  and  more  variety  in 
production.  The  result  is  earlier  delivery  of  the  first  increment,  but  with  later  and  more  costly 
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delivery  of  subsequent  increments  than  if  conducted  via  a  single-step  methodology. 
Curtailments  of  scope  by  the  exclusive  use  of  mature  technology  enable  more  effective  delivery 
of  the  first  increment,  further  illustrated  by  two  case  studies.  Duplication,  rework,  transaction 
costs,  decision  backlog  and  error  are  causes  of  inefficiency  in  the  successive  increments. 
Production  variety  and  mixed  configurations  produce  obvious  implications  for  logistical 
supportability,  training,  failure  causality,  compatibility  and  interoperability,  etc.  Further,  certain 
attributes  of  hardware  products  might  help  determine  the  suitability  of  this  development 
methodology.  Products  that  are  nearly  immutable,  which  have  binary  requirements  for  key 
capabilities,  require  man-rating,  or  are  maintenance-intensive  may  not  be  good  candidates  for 
incremental  development.  Mutable  products  with  costless  production,  continuous  requirements, 
low  maintenance,  or  time  criticality  are  more  likely  to  reap  advantages  from  this  development 
approach.  While  modular  open  systems  architecture  facilitates  system  adaptation,  modularity 
itself  does  not  necessarily  create  evolutionary  advantages  due  to  relative  modular 
interdependency.  Program  managers  must  be  aware  of  the  inherent  risks  of  these  agile 
acquisition  methods  and  take  additional  steps  to  balance  them  with  appropriate  planning  and 
resources,  disciplined  change-control  measures,  organizational  accommodations  and 
accountability  for  configuration  management. 

Keywords:  Evolutionary  acquisition,  spiral  development,  incremental  product 
development.  Javelin,  ATACMS,  agile  development  methodologies,  computational 
organizational  modeling,  modularity. 

Introduction — The  Inevitability  of  Change 

We  are  told  in  Diogenes  Laertius's  Lives  and  Opinions  of  Eminent  Phiiosophers  (early 
3'^^  century)  that  the  Greek  philosopher  Heraclitus  (c.535  -  475  BC)  was  the  first  to  observe  and 
say,  “Everything  flows;  nothing  stands  still,” — ^the  popular  derivation  of  which  is,  “The  only 
constant  is  change.”  Indeed,  everything  does  seem  to  change,  evolve  and  give  rise  to  variety  in 
the  world.  Since  his  work  in  the  1830s,  Charles  Darwin  receives  much  of  the  credit  for 
furthering  a  theory  of  biological  evolution.  While  not  the  first  to  have  the  idea,  he  associated 
observations  of  species  variety  on  the  island  of  Galapagos  with  species  environment,  and 
suggested  that  nature  selected  the  variations  that  were  the  fittest  (Darwin,  1859).  In  its  time 
(and  even  since),  the  idea  was  considered  radical  and  a  threat  to  the  religious  and  social  order 
of  things.  Mere  variety  itself  can  be  controversial,  since,  paradoxically,  variety  is  appreciated  in 
some  domains  (see  the  writings  of  William  Cowper,  1731-1800)^  and  abhorred  in  others 
(Neave,  2000,  March  2)}  At  the  core  of  the  subject  of  evolutionary  acquisition  are  ideas  and 
phenomena  about  variety  and  change.  As  a  policy  for  system  development,  it  is  controversial 
too.  As  with  Darwinian  concepts,  product  evolution  involves  information  transfer,  interaction  with 
the  environment  and  unpredictability  of  change  outcomes.  But  unlike  evolutionary  biology, 
product  variations  and  selections  occur  frequently  and  are  non-random.  Program  managers 
typically  seek  stability — in  program  requirements,  in  funding,  in  system  design,  and  in 
production  configuration.  But  it  seems  the  only  constant  is  change.  Everything  changes  and 


^  See  also:  Kerr  (1979,  p.  65)  about  the  basic  human  need  for  variety  and  complexity.  Ashby’s  Law  of 
Requisite  Variety  states  that  the  internal  regulatory  mechanisms  of  a  system  must  be  as  diverse  as  its 
environment  in  order  to  cope  with  the  variety  of  challenges  imposed  by  it  (Ashby,  1960). 

^  “Variation  is  nasty:  it  makes  things  difficult,  unpredictable,  untrustworthy:  bad  quality.”  “In  a  big  way,  bad 
quality  means  too  much  variation,  good  quality  means  little  variation.” 
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evolves  over  time.  Much  of  what  the  authors  have  found  in  their  foliowing  research  on  spiral 
development  and  project  management  is  about  how  managers  must  cope  with  product  variety 
and  change.  Using  case  study  analyses,  review  of  current  subject  literature,  and  computational 
modeling,  the  focus  of  our  research  was  to  ascertain  the  acquisition  management  implications 
of  spiral  development,  obtain  lessons  learned  in  past  programs  as  applicable  to  future 
development  efforts,  model  and  simulate  projects  using  different  acquisition  approaches,  derive 
predictions  and  make  recommendations  to  project  managers  for  the  effective  and  efficient 
harnessing  and  implementation  of  spiral  development. 

Background 

Projects  have  long  been  defined  as  unique  and  temporary  enterprises,  as  opposed  to 
common  and  ongoing  operations.  The  latest  (2004)  version  of  the  Project  Management  Body  of 
Knowledge  (PMBOK)  increased  its  emphasis  upon  the  term  “progressive  elaboration"  to 
describe  a  third  fundamentai  characteristic  of  all  projects.  It  means,  “developing  in  steps  and 
continuing  by  increments;  worked  out  with  care  and  detail;  developed  thoroughly”  {PMBOK, 
2000;  PMBOK,  2004,  p.  6).  This  term  relates  to  project  uncertainty  and  describes  the  eventual 
realization  of  project  scope  only  after  multiple  iterations  of  pianning.  The  PMBOK  asserts  that 
progressive  elaboration  is  both  a  necessary  characteristic  of  projects  (occurring  throughout  their 
lifecycles),  as  well  as  a  technique  for  development  of  product  specifications.  It  is  accomplished 
via  the  learning  that  takes  place  over  time  as  project  ambiguity  resolves,  so  that  project  scope 
becomes  more  explicit  and  detailed  (as  opposed  to  “requirements  creep,”  which  is  considered 
uncontrolled  change).  The  PMBOK  later  asserts  that  change  in  the  course  of  projects  and 
products  is  inevitabie,  and  mandates  the  need  for  a  disciplined  change-control  process  to 
controi  its  impacts — from  inception  to  completion  {PMBOK,  2004,  p.  119). 

There  are  many  new  DoD  terms  for  project  management  and  product  development 
methods.  DoD  promulgated  evolutionary  acquisition  (EA)  as  policy  in  2000,  and  soon  after, 
spiral  development  for  the  preferred  acquisition  strategy  of  aii  materiei.  EA’s  goai  is  to  phase 
requirements  and  provide  capability  sooner.  But  there  has  been  confusion  over  terms,  despite 
further  elaboration  and  even  codification  in  statute,  and  it  still  persists  today,  along  with  a  lack  of 
full  understanding  of  many  poiicy  impiications — especially  some  inherent  risks.  EA  operationally 
means  there  will  always  be  multiple  product  releases  of  an  item. 

The  policy  thrust  is  primarily  about  the  reduction  of  product  cycle-time  within  an 
uncertain  environment,  by  exciusiveiy  using  mature  technoiogy.  The  DoD’s  requirements 
process  has  aiso  followed  with  “evolutionary”  requirements  documents — a  new  idea. 

Uncertainty  is  the  usual  realm  of  program  managers,  especially  in  defense  systems,  and  is 
usuaily  dealt  with  by  seeking  best  information.  Eariier  reform  initiatives  were  aimed  at 
overcoming  information  gaps  and  technoiogy  lag.  The  1990’s  Integrated  Product  and  Process 
Development  (IPPD)  initiative  was  about  gaining  collective  wisdom  for  early  and  complete 
requirements  realization.  As  concerns  over  DoD  acquisition  program  costs  and  cycle-times 
continue  in  the  current  mid-2000s  era,  the  DoD  has  not  abandoned  the  use  of  IPPD.  But  by 
embracing  evolutionary  requirements  and  acquisition,  it  has  acknowledged  that  information  wiii 
never  be  complete,  either  from  stakeholders  or  with  regard  to  ever-changing  technology.  It  now 
implicitly  concedes  that  developers  will  learn  about  their  design  over  time  (“requirements 
reaiization”),  and  users  wili  accretiveiy  gain  knowiedge  about  how  they  can  better  use  the  new 
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capability  (“product  discovery”).^  Thus,  a  major  paradigm  shift  for  product  development  has 
occurred  in  the  DoD:  from  a  collaborative  quest  to  capture  and  address  all  requirements  early 
on,  to  an  allowance  of  eventual  requirements  discovery  with  full  attainment  only  after 
visualization,  feedback  and  environmental  changes  occur  along  the  way. 

The  Enabler:  Mature  Technology  Reduces  Risk 

This  is  not  to  say,  however,  that  the  DoD  has  in  its  policy  embraced  technological 
uncertainty  for  the  commencement  of  advanced  development.  Quite  the  contrary — for  at  the 
very  heart  of  the  evolutionary  acquisition  strategy  is  the  requirement  for  the  exclusive  use  of 
mature  technology  to  reduce  technology  risk.  The  impetus  for  this  undoubtedly  lies  in  the  body 
of  work  by  the  Government  Accountability  Office  (GAO)  over  the  last  ten  years,'*  which  has 
obviously  and  greatly  influenced  the  DoD  5000  series.  The  GAO  encourages  the  use  of 
knowledge-based  processes  and  specifically  separates  technology  development  from  product 
development.  It  argues  that  shorter  product  cycle-times  are  the  hallmark  of  program  success 
and,  therefore,  should  be  limited  to  five  years  for  more  frequent  introduction  of  new  technologies 
into  weapon  systems,  speeding  them  to  the  warfighter.  We  note  that  this  is  not  much  longer 
than  the  average  development  time  for  a  new  model  of  automobile — typically  3-4  years — which 
occurs  in  a  very  mature  and  cyclical  industry  (Kim,  2002,  June).  The  GAO’s  target  may  ignore 
the  significantly  greater  amount  of  technology  development  required  in  many  DoD  projects 
compared  with  most  automobile  development  projects. 

Most  emphasized  by  the  GAO  (in  the  many  reports  reviewed  by  these  authors)  is  the 
aspect  of  technology  maturity  before  commencement  of  advanced  development.  The  Office 
applies  a  1-through-9  rating  scale  of  technology  readiness  levels  (TRL)  that  was  developed  by 
the  National  Aeronautics  and  Space  Administration,  adopted  by  Army  and  Air  Force  research 
laboratories,  and  recently  implemented  in  the  DoD  5000  series  (in  particular,  the  Defense 
Acquisition  Guidebook — formerly  DoD  5000.2R).  Until  recently,  the  DoD  had  no  specific 
requirements  for  use  of  TRLs,  but  levels  6  and  7  now  satisfy  its  guidelines  for  technology 
maturity  at  Milestone  B.  TRL  6  states  that  the  technology  has  been  demonstrated  in  a  relevant 
environment  (simulating  the  key  aspects  of  the  operational  environment),  and  TRL  7  is  its 
demonstration  in  an  operational  environment  (that  which  addresses  all  operational  requirements 
and  specifications  required  of  the  final  system,  to  include  platform/packaging).  The  GAO  clearly 
prefers  TRL  7  as  the  level  of  technology  maturity  that  will  represent  a  low  and  satisfactory  risk 
for  starting  product  development  (GAO,  2005,  November  15).  The  Office  acknowledges  that 
users  may  not  Initially  receive  the  ultimate  capability  under  this  approach,  but  that  the  initial 
capability  will  arrive  predictably  sooner  and  cheaper  {GAO,  2005,  November  15). 

In  some  respects,  developing  only  mature  technology  as  a  fundamental  program 
requirement  is  similar  to  an  earlier  attempt  to  constrain  project  scope.  Cost  as  an  Independent 


®  The  authors’  terminology  for  what  has  so  often  been  observed  from  their  experiences.  Most  of  us  have 
long  known  that  full  realization  of  requirements  and  visuaiization  of  the  product  often  takes  multiple 
iterations  of  design,  with  feedback  ioops  from  modeiing  and  testing  activities.  And  sometimes  the 
customer  doesn’t  fuiiy  reaiize  what  can  be  done  with  the  product  until  it  is  in  hand.  We  cail  that  product 
discovery,  and  the  authors  can  cite  several  examples  of  this  in  both  commercial  and  defense  applications 
(i.e.,  cell  phones  as  improvised  explosive  device  triggers,  etc.). 

''  See  in  particular:  GAO/NSIAD-98-56;  GAO/T-NSIAD-98-123:  GAO/NSIAD-99-162:  GAO/T-NSIAD-99- 
116;  GAO/T-NSIAD-00-137;  GAO-01-288:  GAO-02-701;  GAO-03-57;  GAO-04-53. 
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Variable  (CAIV)  was  an  acquisition  reform  initiative  that  emerged  in  1995  as  a  means  of  trading 
scope,  or  system  performance,  to  achieve  cost  objectives.  It  was  one  of  very  few  initiatives  that 
were  oriented  on  what,  not  how  (i.e.,  processes)  the  DoD  acquires  its  materiel.^  To  date,  its 
actual  savings  benefit  has  been  difficult  to  quantify,  and  qualitative  measures  have  shown  mixed 
results  (RAND,  2005).  Requirement  attainment  objectives  and  thresholds  were  another  way  to 
facilitate  performance  trades  for  cost. 

When  fully  realized,  it  is  the  exciusive  use  of  mature  technoiogy  in  system  deveiopment 
programs  that  is  the  key  enabier  of  evoiutionary  acquisition  strategy,  facilitating  the  rapid 
transformation  of  applied  technology  to  end-item  capability.  Thus,  it  is  the  third  of  three  principal 
observations,  all  of  which  are  paradigm  shifts,  that  we  have  recently  observed:  (1)  that  the  DoD 
would  now  mandate  program  strategies  for  all  programs  to  have  multiple  product  releases  of  the 
same  item,  (2)  that  requirements  would  be  deferred  or  allowed  to  evolve  over  time,  and  (3)  that 
high  levels  of  technological  maturity  would  be  requisite  for  commencement  of  advanced 
development,  with  an  intended  reduction  of  technical  risk  (and  thus,  project  schedule) 
(USD(AT&L),  2003a,  May  12,  Enclosure— Additional  Policy  El. 14). 

Policy  and  Implementation  Concerns 

But  there  are  questions  and  concerns  about  these  major  shifts  that  several  authors  have 
raised.  Still  a  relatively  new  policy,  observations  and  realizations  about  the  outcomes  of 
evolutionary  acquisition  and  spiral  development  are  only  just  beginning  to  emerge,  and  will 
continue  to  surface  until  at  least  several  major  programs  go  through  their  entire  lifecycle  in  this 
way.  Sylvester  and  Ferrera  (2003)  provided  some  insight  into  the  challenges  and  obstacles  of 
evolutionary  acquisition  implementation — not  from  program-office  level — but  from  the 
perspective  of  strategic  policy-makers  and  subscribers  at  the  Office  of  the  Secretary  of  Defense 
(OSD)-level  during  their  struggle  to  adopt  the  policy.  In  short,  the  authors  explained  the 
aforementioned  confusion  and  ambiguity  of  the  policy  as  it  evolved  from  1983  toward  final 
promulgation  in  2000,  and  then  described  the  conflict  areas  caused  by  shifts  in  power  among 
the  organizational  fiefdoms  in  the  OSD  and  other  affected  institutions  (i.e..  Congress  and  the 
defense  industry).  In  particular,  they  exposed  the  following  major  stakeholder  communities  and 
their  respective  areas  of  concern  about  evolutionary  acquisition: 

•  Congress — loss  of  control  over  DoD  programs  via  specific  and  informed  authorization 
and  approval;  the  inability  to  keep  the  DoD  accountable;  unknown  implications  of 
requirements  and  budget  flexibility  required  for  evolutionary  acquisition. 

•  Military  Departments — need  to  protect  own  acquisition  programs  and  share  of  the  DoD 
budget;  retention  of  funding  for  follow-on  capability  increments;  increased  oversight; 
downstream  logistics  of  multi-configuration  products. 

•  Defense  Industry — disruptions  to  commercial  processes  and  traditional  approaches  to 
business;  competition  for  follow-on  increments;  lower-rate  production  runs  after  shorter 
R&D  efforts. 


®  Some  may  also  assert  that  the  moratorium  against  MILSPECS  was  similar  in  its  thrust  to  reduce 
unnecessary  work  scope,  but  we  believe  specifications  to  be  as  much  prescriptive  (i.e.,  “how”)  as  they  are 
descriptive. 
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•  Comptroller — controlling  programs  and  holding  them  accountabie;  unknown  implications 
of  requirements  and  budget  flexibiiity  required  for  EA  (program  and  budget  “gaming”  by 
services);  “fuil  funding”  policy®  versus  open-ended  requirements  and  fund  streams. 

•  Requirements/Users — sub-optimum  capability;  priority  of  what  is  needed  versus  what  is 
currently  attainable;  loss  of  follow-on  increments. 

•  Test  and  Evaiuation — ioss  of  discipiine  and  assurance  of  operationai  effectiveness  & 
suitability;  iack  of  comprehensive  testing  before  several  low-rate  production 
configurations  are  released. 

We  have  also  had  tactical  (implementation)  concerns  about  excessive  decision 
bureaucracy  (number  of  DAB  reviews — see  Figure  1),  organizational  challenges  from  multiple 
and  concurrent  development  efforts,  outdated  technology  at  release,  funds  forecasting, 
transaction  costs,  and  maintenance  of  subsequent  increment  priority.  It  is  these  phenomena 
that  we  have  modeled  with  computational  organizational  design  tools,  which  will  be  discussed 
later. 


1996  and  2003  DoD  5000  Models 
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Figure  1.  Comparison  of  1996  and  2003  Acquisition  Framework  Models 


®  The  authors  explain  the  dual  meanings  of  this  term  later  in  this  discussion. 
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The  Costs  and  Benefits  of  Variety — and  the  Need  for  Control 

Evolutionary  acquisition  methodologies,  in  addition  to  potentially  adding  more 
concurrency  during  development,  increase  variety  in  production.  Both  concurrency  and  variety 
are  elements  of  complexity  and  program  risk.  Variety  adds  complexity  in  production  and  is 
costly  for  hardware  owners  and  manufacturers  alike.  Traditional  views  about  late  design 
changes  are  negative,  except  for  producibility  enhancements  and  savings  or  correction  of 
design  flaws.  But  market  consumers  often  need  items  in  rapid  cycle-times  and  appreciate 
product  differentiation.  In  support  of  EA  policy,  the  GAO  has  used  product  examples  such  as 
commercial  vehicles.  For  the  most  part,  we  regard  these  commercial  products  as  relatively  “low- 
tech”  on  a  comparative  scale  of  DoD  system  complexity  and  capability.  Moreover,  we  feel  the 
GAO  may  ignore  some  very  important  aspects  of  ownership,  since  the  DoD  is  unique  as  an 
outsourcer  of  capitol  projects  for  internal  use,  and  has  unique  requirements  against  competitive 
threats  in  combat  environments. 

Control  measures  are  used  to  manage  risk.  One  way  of  coping  with  the  complexities  of 
variety  in  ownership  is  via  organizational  and  individual  accountability.  A  recent  example  of 
successful  control  of  rapid  change  lies  in  the  Acoustic  Rapid  COTS  Insertion/Advanced 
Processing  Build  (A-RCI/PB)  program.  In  this  vital  program  for  sustainment  of  submarine 
acoustic  sensing  superiority,  a  series  of  hardware  and  software  upgrades  were  planned  and 
executed  in  rapid  succession.  Each  emerged  with  advancement  in  capability,  keeping  pace  with 
technology  and  competitive  threats,  facilitated  by  rigorous  control  of  interfaces,  standards  and 
protocols  (Boudreau,  2006). 

Many  other  useful  theorems  on  systems  complexity,  change  and  control  exist  that  may 
be  helpful  for  practitioners  to  consider,  but  are  beyond  the  general  scope  of  our  research. 

Do  Product  Attributes  Affect  Spiral  Applicability  and  Outcomes? 

Spiral  development  as  a  universal,  “one-size-fits-aN”  strategy  may  not  always  be 
appropriate.  Perhaps  the  foremost  reservation  is  the  appropriateness  of  the  spiral  development 
process  for  all  project  sizes  and  product  commodities  in  toto,  and  the  application  of  the  spiral 
process  to  hardware  products  versus  Boehm’s  original  and  most  relevant  application  of  this 
development  approach  toward  software.^  We  speculate  whether  certain  product  characteristics 
might  determine  spiral  development  method  applicability,  and,  thus,  may  offer  important 
considerations  for  project  planners. 

•  Mutability  simplifies  change,  and  spiral  development  was  conceived  for  the  most 
malleable  of  products:  “soft”  ware,  which  is  virtually  costless  in  production.  Multiple 
product  increments  do  not  often  appear  in  large,  static,  singular  projects  such  as 
bridges,  highways,  skyscrapers,  or  in  other  project  areas  that  have  typically  long  lead 
times  or  product  cycles,  such  as  feature-length  films,  pharmaceuticals,  etc.  These  are 
what  we  call  nearly  immutable  products  and  are  much  different  than  smaller  projects 
(like  small  application  software  development)  with  much  shorter  development  periods. 


''  And  the  authors  will  be  quick  to  acknowledge  that  software  is  indeed  a  huge  and  growing  part  of 
hardware  systems  large  and  small.  Still,  the  spiral  development  framework  in  current  literature  applies 
overwhelmingly  to  the  realm  of  software,  not  hardware. 
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•  Cycle-time  and  Phase  Concurrency.  Akin  to  relatiyely  mutable  or  immutable  products, 
we  haye  obseryed  the  successiye  product  upgrades  yisible  in  long-running  aircraft 
programs  (UH-60  Blackhawk  and  C-130  Hercules  as  examples)  in  which  there  are 
periods  of  production  configuration  stability,  followed  by  improyement  efforts,  followed  by 
another  stable  use  period.  Cycle-time  for  the  deyelopment  of  each  increment,  and  the 
relatiyely  successive  or  concurrent  phasing  of  the  follow-on  increments,  will  haye  a 
definite  impact  on  program  structure,  budgeting,  project  complexity,  and  organizational 
issues,  etc.  For  reasons  that  we  will  bring  forth  in  our  section  on  the  computational 
modeling  of  spiral  deyelopment,  we  haye  concerns  about  the  conceptualization  of  spiral 
deyelopment  programs  with  continuous  and  highly  concurrent  phasing  of  deyelopment 
increments.  We  suggest  that,  though  concurrency  is  a  necessary  ingredient  for  efficient 
project  management,  it  has  also  long  been  correlated  with  risk  (because  of  actiyity 
interdependency),  and  might  yary  significantly  with  the  types  of  actiyities  underway  (See 
Figure  2) — the  inference  being  that  periods  of  stable  production  configuration  between 
deyelopment  increments  reduce  complexity  in  program  structure  and  attendant  risks. 
Similarly,  shorter  cycle-times  haye  less  opportunity  for  knock-on  effects  or  secondary 
consequences.  Particularly  in  matrix  organization  structures,  as  often  the  case  with 
projects,  there  can  be  a  tendency  to  staff  multiple  projects  with  a  single  specialist.  The 
more  projects  a  specialist  supports,  the  less  they  are  proportionately  ayailable  to  the 
projects  due  to  “queuing  inefficiencies. ’’Ayailability  decreases  because  of  the  need 
for  transition  between  projects  (physical,  mental,  learning  curye,  etc.). 

■  The  end  result  has  sometimes  been  shown  to  be  large  delays  in  project  completion 
(Smith  &  Reinhartsen,  1998). 

■  Similarly,  Ibrahim  (2005)  has  shown  that  discontinuous  enterprise  membership  is  a 
contributing  factor  toward  knowledge  loss  in  organizations  inyolyed  in  large,  complex 
product  deyelopment  processes.  Examining  knowledge  flows  across  product 
lifecycles,  members  often  are  not  engaged  in  all  phases.  Whether  from  rotation  of 
duties  or  multi-tasking,  a  discontinuous  member’s  inaccurate  knowledge  could  cause 
a  functional  error  at  the  indiyidual  leyel  which  is  not  obyious  at  the  enterprise’s 
oyerall  project  leyel.  These  findings  support  obseryations  of  knowledge  loss 
continuing  despite  inyestments  in  information  technology  and  knowledge 
management. 
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Figure  2.  Concurrency  Relative  to  Types  of  Activity 

•  User  Risk  (Safety  and  Time  Criticality).  Time  criticality  and  life-saving  dependency,  as 
opposed  to  user  hazard  levels  (safety  &  man-rating),  might  seem  to  also  have  influence 
over  design  approaches.  We  have  discussed  above  the  area  of  technological  risk  and 
the  DoD’s  use  of  incremental  or  spiral  approaches  to  resolve  it  (along  with  a  compulsory 
policy  for  the  advanced  development  of  only  relatively  mature  technology).  But  DoD 
products  have  expanded  risk  considerations  beyond  Boehm’s  models  of  commercial 
software.  Extending  the  idea  of  project  risk-as-a-driver  down  to  the  level  of  the  end-user, 
it  might  seem  logical  to  assume  that  time  criticality  of  the  need  or  mission,  where  risk  of 
not  achieving  project  success  actually  endangers  customer  lives,  might  be  a  significant 
factor  in  the  appropriate  application  of  the  spiral  process  for  reduced  initial  product  cycle¬ 
time.  Perhaps  defensive  systems  are  a  good  example.  The  immediate  needs  for  a 
Rocket-Propelled  Grenade  (RPG)  defeateroran  Improvised  Explosive  Device  (lED) 
neutralizer  for  currently  deployed  forces  in  Iraq  and  Afghanistan,  for  example,  clearly 
dictate  that  lives  will  be  lost  if  a  near-term  capability  is  not  achieved.  We  also  cite  as  an 
example  the  National  Missile  Defense  (NMD)  initiative,  in  which,  in  view  of  near-term 
threats,  early  deployment  of  even  rudimentary  capability  has  been  deemed  preferable  to 
waiting  for  full  capability.  Such  urgency  likely  precludes  full  and  certain  requirements 
specificity. 

■  Non-man-rated  Systems:  In  an  almost  opposite  vein,  non-man-rated  systems,  such 
as  Unmanned  Aerial  Vehicles  or  cave-exploring  robots — capabilities  in  which 
operator  lives  are  not  at  risk  if  the  product  fails — may  also  lend  themselves  readily  to 
rapid  innovation  and  risk-less  experimentation  cycles.  However,  user  hazard  levels 
for  man-rated  systems  may  be  a  different  matter. 
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■  Man-rated  Systems:  Configuration  variety  adds  technical  complexity  with 

unpredictable  interactions.  In  such  projects  as  pharmaceuticals,  aviation,  vehicular 
transportation,  etc.,  producers  mitigate  safety  risks  with  thorough  analyses, 
documentation  reviews,  testing  and  other  control  and  verification  processes.  By  their 
very  nature — ^with  lethal  hazards  for  the  end-user,  and  typically  lengthy  approval 
requirements — these  may  not  be  good  candidates  for  a  spiral  approach.  We  believe 
this  is  why  space  experts  say  they’ll  not  use  spiral  development  with  NASA’s  new 
Crew  Exploration  Vehicle  (Roy,  2006). 

•  Production  Quantity.  As  to  product  size  or  production  quantity,  we  find  no  evidence  that 
either  precludes  use  of  spiral — as  with  space  vehicles  and  large  ships — though  support 
considerations  do  arise  with  variety  that  could  greatly  affect  total  costs  of  ownership. 

•  Logistical  Support  planned  during  Service/Shelf  Life.  Our  observations  warn  that  multiple 
configurations  of  hardware  products  do  come  at  a  cost  for  ownership.  Veterans  of  new 
system  deployments  across  the  force/fleet,  or  throughout  any  large  using  organization, 
know  the  difficulties  of  rolling  out  a  configuration  change.  Benefits  of  standardization 
have  long  been  offered  via  production  economies  of  scale,  commonality  of  parts  across 
platforms,  and  interoperability.  If  the  ultimate  goal  is  to  have  standardization  across  the 
DoD’s  force,  owning  multiple  configurations  of  a  system  (variety)  equates  to  added 
complexity  in  training  and  supply  support  of  the  item.  Neither  can  the  logistical 
maintenance  strategy  be  ignored:  whether  the  end-item  is  maintenance-intensive  (such 
as  tactical  vehicles)  or  maintenance-free — such  as  with  many  electronics  items  and 
munitions,  and  situations  in  which  physical  changes  are  completely  transparent  to  the 
user.  For  multiple  product  configurations,  the  answer  could  have  a  huge  effect  on  the 
total  costs  of  ownership,  as  shown  by  RAND  on  the  proliferation  of  UAVs  (Shaver  & 
Amouzegar,  2005). 

•  Range  of  Reguirement  Attainment.  Certain  requirements  are  binary  rather  than 
continuous.  Examples  are  soft  launch,  network  security,  physical  fit,  leak-proof, 
shock/vibration/drop  proof,  survivability,  horizontal-to-vertical  flight  transition,  etc.  If  one 
of  these  more  binary-type  requirements  happens  to  be  a  key  performance  parameter,  its 
attainment  will  be  on  the  project’s  critical  path  and  highly  dependent  upon  technical 
maturity.  As  such,  it  may  practically  dictate  the  length  of  the  entire  advanced 
development  effort  and  make  division  into  capability  increments  less  beneficial  as  a 
development  strategy. 

•  Amount  of  Change — and  the  Lure  of  Modularity.  These  authors  subscribe  to  the  current 
theorists’  view  that  system  complexity  is  comprised  of  numbers  (of  components), 
connections  (interdependencies)  and  distinctions  (variety).  Distinction  corresponds  to 
variety,  to  heterogeneity,  and  to  the  fact  that  different  parts  of  complex  systems  behave 
differently  (Heylighen,  1997).  Variety  is  a  component  of  Nobel  Prize  winner  Herbert 
Simon’s  explanation  of  complexity — many  different  parts  with  many  interactions.  Simon 
argues,  from  his  observation  of  complexity  in  things  both  natural  and  artificial,  that 
complex  systems  evolve  from  simple  systems.  And  they  do  so  more  rapidly  when  there 
are  stable,  intermediate  forms  or  sub-systems  (like  modules  or  “units  of  action”)  (Simon, 
1981 ).  While  the  concept  of  modularity  suggests  approximately  independent  subsystems 
may  be  modified  or  adapted  as  such,  it  has  been  shown  that,  in  the  aggregate,  there  is 
yet  quantifiable  modular  interdependency  that  affects  evolvability  (Watson  &  Pollack, 
2005).  In  other  words,  how  changes  in  the  state  of  one  module  affect  the  state  of 
another  is  relative  and  measurable.  Thus,  we  suggest  it  is  not  only  the  focus  upon 
structural  modularity  as  such,  and,  standard  interfaces,  that  enable  systems  evolution. 
Rather,  it  is  the  relative  interdependency  of  the  modules.  In  short,  PMs  need  to  be 
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mindful  of  the  degree  of  change  in  subsequent  spirals.  One  subsystem  is  likely  to  affect 
another  in  the  short-  or  long-run.  And  that  can  make  product  evolution  problematic.  As 
Norman  Augustine  once  said,  “No  change  is  a  small  change”;  independent  subsystems, 
even  redundant  ones,  aren’t  always  independent  (Augustine,  1997,  June). 

Development  Case  Studies 

One  of  the  most  recent  monographs  we  have  found  on  emerging  results  of  evolutionary 
acquisition  is  by  RAND — on  five  immature,  non-man-rated  space  systems.  Space  systems  are 
somewhat  different  (in  quantities,  space  environment,  front-end  investment,  and  extended 
technology  development  periods)  than  other  DoD  sytsems.  RAND  also  found  that  policy 
confusion  persists,  and  that  EA  added  program  complexity  and  uncertainty,  especially  with 
regard  to  budgeting.  Extending  their  findings  to  non-space  DoD  programs,  RAND  highlighted 
the  EA  challenges  of  programmatic  flux.  They  feel,  and  we  agree,  that  EA  presents  the 
opportunity  for  typical  PM  challenges  to  be  even  more  formidable. 

Two  missile  programs  were  used  as  case  studies  for  analysis  and  to  illustrate  planned 
and  unplanned  change.  ATACMS  used  incremental  and  spiral  strategies  for  product 
development.  The  program  skipped  its  technology  development  phase  by  employing  mature 
technologies  for  a  leap-ahead  capability  in  range.  It  arrived  on  budget  and  schedule,  with 
several  successive  variants,  pre-planned  and  unplanned.  One  instance  of  production  change 
caused  missile  failure  and  costly  refit  of  already  produced  missiles — underscoring  the  need  for 
more  thorough  design  specification  and  configuration  management  accountability. 

Javelin  used  the  single-step-to-full-capability  approach  to  product  development.  The 
program  embarked  upon  advanced  development  with  immature  technologies  in  several  critical 
areas,  causing  significant  cost  and  schedule  overruns.  It  also  has  had  subsequent  design 
changes  and  product  variety,  more  so  as  running  production  changes  than  as  product  variants. 

Synthesis  of  these  cases  conveys  that  as  an  approach  oriented  primarily  for  reduction  of 
product  cycle-time,  incremental  or  spiral  development  can  successfully  be  used  when 
developing  mature  technologies  first.  But  a  system’s  physical  properties  like  mutability,  along 
with  other  factors  such  as  time  criticality  (user  risk)  and  modular  interdependency,  will  drive 
spiral  development  applicability.  And  key  capabilities  may  in  fact  depend  upon  the  least  mature 
technologies  or  even  binary  requirements.  An  “open,”  or  at  least  elegant,  architecture  is  key  to 
forming  a  basis  for  independent  modular  variety;  and  thorough  design  specification  and 
configuration  management  accountability  is  essential  for  managing  the  complexity  of  multiple 
product  releases.  All  amorphous  spirals  eventually  become  defined  increments.  Other  well- 
known  programs  have  used  a  spiral  approach  over  their  long  product  life  spans,  but  often  have 
successive  phasing  of  their  development  increments. 

Computational  Modeling  of  Spiral  Development 

A  computational  experimentation  approach  to  investigating  evolutionary  acquisition 
projects  is  explored  below.  This  approach  integrates  theory  and  practice  in  a  computational  tool 
that  allows  controlled  experimentation  through  simulation.  The  current  work  reflects  project 
theory  (e.g.,  the  theory  of  constraints  and  work  flows),  product  development  theory  (e.g.,  rework 
impacts  and  work  dependencies),  and  management  (e.g.,  resource  management  and 
information  theory).  Practice  is  reflected  in  the  model  through  the  use  of  case  studies  to  build 
and  validate  the  model  structures  (as  described  in  the  literature  cited)  and  the  calibration  and 
testing  using  the  acquisition  projects  described  above.  A  computational  experimentation 
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approach  provides  many  advantages  over  pure  and  benefits  from  several  of  the  strengths  of 
both  laboratory  and  field  research.  Nissen  and  Buettner  (2004)  describe  and  discuss  the 
computational  experimentation  approach,  and  Dillard  and  Nissen  (2007)  describe  its  application 
to  investigating  acquisition  projects. 

The  system  dynamics  methodology  was  applied.  System  dynamics  uses  a 
computational  experimentation  approach  to  understanding  and  improving  dynamically  complex 
systems.  The  system  dynamics  perspective  focuses  on  the  roles  of  accumulations  and  flows, 
feedback,  and  nonlinear  relationships  in  managerial  control.  The  methodology’s  ability  to  model 
many  diverse  system  components  (e.g.,  work,  people,  money),  processes  (e.g.,  design, 
technology  development,  quality  assurance),  and  managerial  decision-making  and  actions  (e.g., 
forecasting,  resource  allocation)  make  it  useful  for  investigating  acquisition  projects.  Forrester 
(1961)  develops  the  methodology's  philosophy,  and  Sterman  (2000)  specifies  the  modeling 
process  with  examples  and  describes  numerous  applications. 

Modeling  Incremental  Development  with  Multiple  Development 
Blocks 

Figure  3  depicts  an  acquisition  project  with  multiple  increments  or  blocks.  Subsequent 
blocks  have  the  same  basic  information  flow,  but  can  also  be  delayed  by  the  completion  of 
phases  in  previous  blocks  or  constrained  by  the  progress  in  their  own  blocks.  Importantly,  in 
addition  to  the  flow  of  information  downstream  through  phases  (black  arrows  in  Figure  3), 
multiple  iteration  acquisition  also  provides  opportunities  for  information  to  flow  upstream,  such 
as  from  User  Product  Testing  in  an  earlier  iteration  to  Develop  Requirements  or  Advanced 
Development  in  a  subsequent  iteration  (red  vertical  arrows  in  Figure  3). 
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Figure  3.  Information  Flows  in  an  Incremental  Acquisition  Project 

In  the  model,  the  structure  of  each  block  Is  the  same,  although  parameter  values  are 
varied  to  reflect  different  acquisition  projects  and  strategies.  For  example,  all  phases  include 
start-up  work  that  is  not  directly  applied  to  generating  development  products  (requirements, 
technologies,  component  designs,  or  products).  Each  phase  also  includes  the  requisite  review 
work  that  also  does  not  directly  generate  product.  This  is  consistent  with  GAO 
recommendations  to  manage  each  development  block  like  an  individual  project.  One  impact  of 
this  loading  of  each  phase  with  start-up  and  review  work  that  we  suspect  has  only  been 
recognized  informally  is  a  significant  increase  in  the  total  amount  of  work  required  to  provide  a 
given  set  of  requirements  to  warfighters  when  multiple  development  blocks  are  used.  As  was 
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shown  with  our  modeling  results,  this  work  has  a  significant  impact  on  project  performance  that 
may  impact  the  types  of  projects  in  which  spiral  development  can  be  effective. 

Computational  modeling  of  incremental/spiral  versus  a  single-step  methodology  yields 
results  that  illustrate  our  implementation  concerns.  Spiral  development  can  provide  the  initial 
increment  delivery  with  some  (but  not  all)  requirements  satisfied  earlier  than  in  single-block 
development.  However,  spiral  development  takes  more  time  and  costs  more  to  satisfy  all 
requirements  than  single-block  development.  Spiral  development  has  a  high  risk  of  not 
satisfying  all  requirements  by  the  time  single-block  development  can  satisfy  all  requirements 
(See  Table  1). 
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Table  1.  Performance  Comparison  of  Three  Simulated  Acquisition  Projects 

The  causal  paths  that  drive  and  constrain  project  performance  in  spiral  development 
pass  through  multiple  types  of  resources,  development  processes,  and  move  across  both 
development  phases  and  development  blocks.  They  also  vary  widely  for  different  performance 
measures.  This  makes  the  drivers  of  and  constraints  on  spiral  acquisition  project  performance 
more  difficult  to  identify  than  those  in  single-block  development  projects.  Our  modeling  results 
indicate  that  spiral  development  is  a  significantly  different  approach  to  acquisition  than  single¬ 
block  development,  and  requires  different  planning,  resourcing,  and  management. 

The  concurrent  use  of  multiple  development  blocks  in  spiral  development  significantly 
increases  the  number  of  development  phases  and  activities  that  must  be  managed  and 
coordinated  at  any  given  time  compared  to  single-block  development.  This  increases  the  project 
management  needs  for  successful  acquisition  in  spiral  development  projects  compared  to 
single-block  projects. 

As  in  single-block  development,  progress  in  spiral  development  requires  the 
identification  and  understanding  of  progress  bottlenecks.  The  concurrence  and  resulting 
complexity  of  development  in  spiral  projects  causes  the  types  and  locations  of  bottlenecks  to 
vary  widely  and  be  more  difficult  to  identify  and  address  than  in  single-block  development. 
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Causal  paths  of  the  drivers  and  constraints  on  project  performance  and  progress 
bottlenecks  move  from  one  feature  of  a  project  to  another  as  projects  evolve.  The  increased 
dynamics  of  development  in  spiral  development  projects,  as  compared  to  single-block 
development,  make  identifying  and  addressing  causal  paths  and  progress  bottlenecks  more 
difficult.  Progress  bottlenecks  can  cause  counterintuitive  behavior,  such  as  reductions  in  project 
cost,  by  adding  resources  at  a  bottleneck.  Understanding  and  exploiting  the  opportunities 
provided  by  these  behaviors  requires  a  deep  understanding  of  the  project  structures  and 
dynamic  interactions  that  drive  and  constrain  progress. 

Discussion — Recent  Views  on  Balancing  Risk  and  Control 

Boehm’s  latest  book  on  software  development  advocates  balancing  disciplined  and  agile 
methods  to  capitalize  on  the  benefits  of  both.  Discipline  is  needed  as  a  control  mechanism  to 
avoid  risk,  but  agility  is  needed  to  respond  quickly  to  customer  needs.  Saying,  “One  size  fits  all 
is  a  myth”;  he  advocates  a  balanced  approach  based  upon  risk.  He  also  advocates  the  more 
disciplined,  risk-averse  approaches  for  projects  that  are  mission/safety  critical,  larger  in  size, 
and  have  more  stable  requirements  (Boehm,  2004). 

It  could  be  summarized  that  spiral  development  was  at  its  inception  and  is  at  its 
extension  all  about  risk.  Paradoxically,  it  is  an  agile  method  envisioned  to  reduce  risk  and,  yet, 
can  potentially  add  its  own.  On  the  one  hand,  a  spiral  or  incremental  approach  allays  risk  by 
reducing  scope  to  render  only  the  highest  priority  capabilities  with  the  exclusive  use  of  mature 
technology,  and  obtains  early  and  continuous  feedback  from  the  environment  for  follow-on 
developments.  On  the  other  hand,  it  introduces  concurrency  during  advanced  development  and 
adds  variety  in  production,  with  all  their  attendant  management  challenges. 

Observations  and  Assessments 

Although  today’s  policy  of  evolutionary  acquisition  is  prescribed  as  a  development 
methodology,  it  is  actually  focused  more  upon  what — not  how — \Ne  develop.  As  such,  it  is  about 
doable  scope,  reducing  risk  via  exclusive  use  of  mature  technology.  The  Cost  As  an 
Independent  Variable  and  other  requirement-limiting  initiatives  were  earlier  attempts  to 
accomplish  this,  by  encouraging  product  performance  trades  to  keep  cost  estimates  fixed.  As 
with  CAIV,  this  likely  means  trading  performance  requirements  for  earliest-deploying 
increments. 

Spiral  development  also  seeks  to  spread  out  the  technical  risk  over  more  development 
and  process  time  via  incrementing.  We  have  shown  with  simulation  that  this  can  potentially 
improve  risk  management  performance  initially,  but  with  higher  overall  costs  and  longer 
subsequent  development  durations,  if  deliberately  deferring  known,  estimable  work.  As  such, 
our  computational  modeling  indicates  that  incremental  development  costs  more  and  requires 
more  time  to  provide  the  same  requirements  than  single-step  development.  With  regard  to 
project  risk,  the  increased  complexity  in  a  project  using  an  incremental  or  spiral  approach 
makes  the  isolation  and  effective  management  of  progress  bottlenecks  more  difficult  than  in 
single-step  development. 

The  policy  change  is  that  spiral  development  now  includes  undefinitized  increments  and 
prescribes  incremental  development  instead  of  single  step  development.  All  amorphous  spirals 
will  eventually  become  defined  increments — mini-programs.  In  years  past,  they  have  often  been 
implemented  as  sequential,  separate,  and  successive  product  upgrades  (such  as  the  CH-47, 
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UH-60,  C-130,  B-52  program  examples).  But  current  policy  expresses  these  as  more 
concurrent,  frequent  and  continuous.  Such  concurrency  adds  complexity  to  development 
models,  with  attendant  risks  of  over  allocation  of  work,  noise,  error,  duplication,  and  other 
inefficiencies  from  work  deferral  and  divided  effort  in  project  management  organizations. 
Additional  oversight,  reviews,  contracting,  testing,  etc.,  will  also  likely  affect  transaction  costs.  If 
all  requirements  are  known  and  an  incremental  approach  is  used,  then  there  is  a  deliberate 
deferral  of  work  to  later  increments,  and  there  will  be  a  resultant  Increase  in  total  development 
costs  and  durations  for  these  same  reasons. 

Recommendations  for  Practice 

1 .  Project  managers  need  to  be  aware  of  the  inherent  risks  of  spiral  development  and  take 
necessary  precautions  to  balance  those  risks.  Many  tools  and  control  measures  are 
currently  developed  and  available  to  assist  project  managers  in  balancing  the  risks  of 
spiral  development,  such  as  technology  readiness  levels,  configuration  management, 
technology  performance  management,  real  options,  project  phasing,  risk  management, 
earned  value  management  and  organizational  design. 

2.  Incremental  and  spiral  development  projects  provide  additional  opportunities  for 
managing  development  risk  in  the  project  design.  These  include  project-planning 
decisions  about  the  number  of  development  blocks,  the  requirements  and  associated 
technologies  and  design  components  to  be  included  in  specific  blocks.  This  planning 
provides  opportunities  to  anticipate  where  critical  progress  bottlenecks  may  occur  and 
design  how  to  best  monitor  potential  bottlenecks  and  respond  to  them. 

3.  Product  attributes  may  help  determine  the  suitability  of  spiral  development  as  the  best 
methodology.  PMs  may  wish  to  consider  such  characteristics  as:  mutability,  time 
criticality,  man-rating,  modular  interdependency,  key  parameters  of  capability  versus 
range  of  requirement  attainment  (i.e.,  binary  vs  continuous),  and  the  relative  amount  of 
concurrency  among  increments. 

4.  Progress  bottlenecks  in  iterative  and  spiral  development  often  oscillate  between  process 
constraints  (e.g.,  availability  of  work  due  to  upstream  progress)  and  resource  constraints 
(developer  or  project  management  quantities  or  productivities).  Successfully  addressing 
a  constraining  progress  bottleneck  often  shifts  the  limit  on  progress  to  a  different  location 
in  the  project.  Therefore,  a  structured  and  interdisciplinary  practice  of  identifying  and 
addressing  bottlenecks  can  improve  performance. 

5.  Configuration  management  accountability  must  be  assigned  or  kept  to  maintain 
supportability  and  failure-mode  identification  and  causality  and  prevent  the  variety 
generated  by  spiral  development  from  reducing  total  product  performance. 

Conclusions 

We’ve  suggested  that  a  one-size-fits-all  methodology  for  DoD  system  development  may 
not  be  appropriate,  and  have  offered  for  consideration  several  product  attributes  that  might  help 
determine  the  efficacy  of  the  spiral  approach.  We  further  suggest  that  spiral  development  may 
serve  better  than  single-step  development  for  Initial  capability  when  products  are  mutable,  time- 
critical,  non-maintenance  intensive,  and  have  continuous  (vs.  binary)  or  uncertain  requirements, 
short  cycle-times  (less  knock-on  effects),  sequentially  phased  development,  and  modular 
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independence.  In  contrast,  spiral  development  may  not  be  appropriate  when  there  are  safety  or 
man-rating  concerns  and  have  attributes  opposite  to  those  above.  In  particular,  PMs  should 
understand  the  nature  of  their  product  requirements  with  regard  to  their  range  of  attainment  and 
relative  to  key  parameters  of  capability,  and  vis-a-vis  the  readiness  level  of  their  enabling 
technologies.  Some  key  features  may  indeed  be  binary,  and  others  may  have  significant 
ramifications  of  partial  attainment — such  as  propagated  change  across  the  entire  product 
componentry  (as  in  weight  reduction),  versus  a  more  independent,  modular  modification. 

Open  design  standards  will  not  always  be  incorporable.  And  product  variety  will  emerge, 
with  and  without  backward  compatibility,  interoperability,  etc.  Variety  is  both  an  asset  (for  end- 
users)  and  a  liability  (for  manufacturers,  owners  and  supporters).  As  such,  to  compensate  for 
product  variety,  “owners”  must  “own”  the  design  and  emphasize  configuration  management, 
keeping  or  assigning  responsibility  for  that  function,  and  maintaining  accountability  ior  it. 

Both  product  specifications  as  well  as  risk  realization  in  spiral  development  move  from 
being  amorphous  to  defined.  Spiral  development  has  inherent  challenges,  both  strategic  and 
tactical,  of  which  PMs  must  be  aware.  We’ve  highlighted  and  illustrated  them  here,  as  well  as 
showing  (in  our  case  studies)  that  spiral  development  can  indeed  work — especially  for 
technically  mature  and  mutable  products  with  open  or  elegant  architecture.  Program  Managers 
must  be  aware  of  these  inherent  risks,  and  take  necessary  precautions  to  balance  them  with 
increased  use  of  tools,  such  as  technology  readiness  levels,  configuration  management, 
technical  performance  measurement,  contract  incentives,  options  and  phasing,  organizational 
design,  etc. 

Stability  is  the  quest  in  all  things  programmatic — for  funding,  requirements,  design, 
configuration,  etc.  But  in  an  unstable  world,  and  with  the  future  being  necessarily  uncertain,  the 
tension  between  control  and  change  is  probably  unending.  PMs  do  have  some  tools  for  coping, 
and  being  forewarned  is  being  forearmed.  PMs  are  used  to  concurrency  and  change,  as  they 
are  largely  what  make  project  management  what  it  is — a  balancing  act.  Mechanisms  for  control 
of  risk  include  project  management  tools  such  as  configuration  management,  technical 
performance  measurement,  earned  value  management,  risk  management,  etc.  Organizational 
and  cultural  factors  such  as  leadership,  trust  and  accountability  play  a  significant  role  as  well. 
Successful  use  of  these  tools  to  balance  control  and  risk  in  projects  with  a  high  rate  of  change 
and  concurrency  is  an  area  for  our  further  study. 
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■  Financing  DoD  Budget  via  PPPs 

■  ROI  of  Information  Warfare  Systems 

■  Acquisitions  via  leasing:  MPS  case 

■  Special  Termination  Liability  in  MDAPs 
Logistics  Management 

■  R-TOC  Aegis  Microwave  Power  Tubes 

■  Privatization-NOSL/NAWCI 

■  Army  LOG  MOD 
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PBL  (4) 

Contractors  Supporting  Military  Operations 
RFID  (4) 

Strategic  Sourcing 

ASDS  Product  Support  Analysis 

Analysis  of  LAV  Depot  Maintenance 

DiffusionA/ariability  on  Vendor  Performance  Evaluation 

Optimizing  CIWS  Lifecycle  Support  (LOS) 


Program  Management 


Building  Collaborative  Capacity 

Knowledge,  Responsibilities  and  Decision  Rights  in  MDAPs 
KVA  Applied  to  Aegis  and  SSDS 

Business  Process  Reengineering  (BPR)  for  LCS  Mission  Module  Acquisition 
Terminating  Your  Own  Program 
Collaborative  IT  Tools  Leveraging  Competence 


A  complete  listing  and  electronic  copies  of  published  research  within  the  Acquisition 
Research  Program  are  available  on  our  website:  www.acauisitionresearch.orq 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  EOR INEORMED  CHANGE 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OE  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 

555  DYER  ROAD,  INGERSOLL  HALL 

MONTEREY,  CALIEORNIA  93943 


www.acquisitionresearch.orq 


'Acquisition  Research,  Pxo^m: 
Creating  Synergy  for  Informed  Change^ 


-  ^ 


i  ^ 


Too  Little  Too  Soon? 

Modeling  the  Risks  of  Spiral  Development 


John  Dillard 

Naval  Postgraduate  School 


David  Ford 

Texas  A&M  University 
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"Evolutionary  Acquisition"  Is  a  Promising 
Strategy,  But  Has  Been  Difficult 
to  Implement 

In  2003,  I  lie  LLS.  DepiiiL  iTieni  ol  DcJeusc  (DoO)  specilied  cTOlntioiiiin^  ncquisirlon  (EA)  as  tlie 
prcJrciicd  ap[ToacJi  ro  weapon  sj'srcni  acquisirioii,  and  spiiiTl  dc\t'lopinciit  as  tlic  prefened  nicaiis  of 
iinplemeiirLirioii.  EA  strategies  ltUit  to  tlev^lop  tievi^  capnblJlties  in  JiiLiltijilc  increjnciits,  as  opposed  to 
ilie  triTtiLtional  strateg}'  ol  developing  a  inJJ  capability  in  a  single,  leiigtliier  srep>,  EA  strategies  are  Jiieatit 
to  reduce  ibe  linie  il  takes  to  Held  opera tionalK'  nschil  equipment,  control  technical  risk  and  cost  growth, 
and  make  cost  est'iiiiates  more  reliable  for  each  stage  of  developmejit,  while  alio  whig  greater  Hexihility 
to  ev'ahiate  and  improve  a  program  based  on  experience  in  the  Hekh  lliis  greater  llcxibility  arises  in  part 
from  the  hict  that,  with  the  spiral  development  approach,  the  end-state  requirements  are  not  known  at 
program  in  ilia  Lion,  ['■iii  rather  emerge  an(.l  evolve  through  an  iterative  process  of  phaseed  deTt'elopjnent  and 
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“Evolutionary  acquisition 
strategies  shali  be  preferred 
approach  to  satisfying 
operationai  needs.” 

DoDI  5000.2 


Evolutionary  Acquisition  Model 


CD 

O 

c. 

CO 


CD 

Q_ 


Block  III  Requirements 


Block  II  Requirements 


Evolutionary  Acquisition 


Increment  2 


Increment  3 


Further  defined: 

*  Incremental  Development:  A  desired  capability  is  identified;  the 
end-state  requirement  is  known:  and  that  requirement  is  met  over 
time  by  developing  several  increments,  each  dependent  on 
available,  mature  technology. 

*  Spiral  Development:  A  desired  capability  is  identified,  but  the  end- 
state  requirements  are  not  known  at  program  initiation. 
Requirements  are  refined  through  demonstration  and  risk 
management;  there  is  continuous  user  feedback;  and  each 
increment  provides  the  user  the  best  possible  capability. 


lopment  and 
future  product 


Key 


Time 


^  Technology  ^ 

^User 

Feedback 

Feedback 
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“Paradigms  influence  our  study 
between  revolutions”  Kuhn 
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Barry  Boehm’s  Spiral  Model  of 
Software  Development 


Evaluate 


Identify 
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Evaluation 
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Requirements 


Evaluation 


Risk 

Analysis 


System 

Requirements 


Proof  of 
Concept 
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Requirements 


Coticepiual 

Design 


Logical 

Design 
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Build 


Physical 

Design 


Final 

Build 


Final 

Design 


Construct 


Design 


PMI’s  PMBOK: 


A  Guide  to 

project  Manageirtent 
of  Knowledge 

Third  Edition 

(PMBOK*  Guide) 


STANDARD 


“Progressive  Elaboration 
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Evolutionary  Acquisition  Issues 

•  Number  of  OSD-Level  Reviews 

-  Off-Core  Activities 

-  Significant  Transaction  Costs 

•  Unplanned  work  is  inestimable. 

•  Fielding  of  obsolete  technology  -  if  SDD  isn’t  short 

•  Continued  conceptual  and  definitional  ambiguity  (RAND) 

•  increment:  Militarily  useful  vs.  all  desired  capabilities 

•  Organizational  impacts  of  concurrent  production  and  development  of 
follow-on  increments 

•  Maintaining  of  funding  priority  for  follow-on  increments 

•  GAO  examples  are  mostly  from  cyclical  commercial  models,  versus 
fleet  ownership  (i.e.,  United,  UPS,  Fedex) 

•  Variety  brings  benefits  and  costs 


Everything  Changes,  But... 


Adobe  Reader  7.0.9  for 
Windows  2000  SP2-SP3, 
English 

Latest  version 


A  one-size-fits-all  development 
methodology  may  not  be  appropriate 
for  all  product  commodities. 
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Product  Attributes  May  Affect  the 
Development  Strategy 

Mutability 

Range  of  Requirement  Attainment  (Binary  vs.  Continuous) 
User  Risk  (Safety  and  Time  Criticality) 

-  Time-critical  or  enhanced  survivability  systems  (NMD,  ARCI) 

-  Non-man-rated  Systems  (UAVs) 

-  Man-rated  Systems  (munitions) 

-  Production  Quantity  (not  a  factor) 

Logistical  Support  Planned  During  Service/Shelf  Life 
Net  Amount  of  Change  -  and  the  Lure  of  Modularity 

-  Changes  propagate  with  relative  modular  interdependency 


Relative  Concurrency  of  Increments 


■H 

L_ — J 

f - ^ 

- j 

iSSS 

4  ' 

Development  Increments  Concurrent  with  Later  Production 


U  «!■  il#  IP 

li 

1 

4? — — d 

Deveiopment  increments  Concurrent  with  Initial  Production 
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A  Tale  of  Two  Missiles 


Spiral  and  Incremental  Development 


Single  Step  to  Full  Capability 


ARMY  TACMS 

MISSILE  DESIGNED  FOR  GROWTH  WARHEADS 


M74  Submunrtfon 


ALTERNATIVE  BLOCK  II 
WARHEADS  CONTAIN 
"SMART"  ANTI- ARM  OR 
SUBMUNITIONS 


MISSILE  DESIGN 
OPTIMIZED  FOR 
BLOCK  II  PAYLOAD 
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A  Tale  of  Two  Missiles: 


Technology  Maturity  -  A  Key  Difference 


Key  Program  Characteristics  -  First  Increment  of  Capability 

Proqram  Aspects 

AT  AC  MS 

JAVELIN 

DARPA  Predecessor 

Assault  Breaker  1977-82 

Tank  Breaker  1981-82 

Ultimate  Capability 

“Deep  Attack” 

“Fire  <S  Forgef 

Critical  Technoloqies  &  Readiness  Levels: 

Munition 

9  -  Lance  M74  Bomblet 

5-  Tandem  Shaped  Charges 

Propulsion 

9  -  Solid  Rocket  Motor 

5-  Two-Stage  Solid  Rocket  Motor 

Flight  Control 

9  -  Fin  surfaces 

6  -  Fins  +  Thrust  Vector  Control  Vanes 

Guidance  and  Control 

9  -  Inertial 

4  -  Tracker  Software  Algorithm 

Safe/Arm  Fusing 

7  -  Mechanical 

4-  Electronic 

Software  Function  (Target Acquisition,  Fire  Control,  etc.) 

6  -  Various 

6-  Various 

Sensor 

N/A 

5  -  Focal  Plane  Array 

Capability  Leap  Area 

Range 

Range,  Lethality,  Survivability 

Cost  of  development 

-$700M 

-$700M 

Contract  Type 

Fixed  Price 

Cost  Reimbursable 

Tech  Development  Phase 

0  Months 

27  Months 

Advanced  Development  Phase  -  Planned 

48  Months 

36  Months 

Advanced  Development  Phase  -  Actual 

51  Months 

54  Months 

Total  Time  in  Development 

51  Months 

81  Months 

Advanced  Development  Phase  Contract  Cost  Growth  0% 

>150% 

Modeling  the  Drivers  and  Impacts  of 
Evolutionary  Acquisition 


•  Need  to  validate  the  impacts  of  Evolutionary  Acquisition 
suggested  by  the  ATACMS  and  Javelin  cases 

•  Need  to  identify  other,  less  clearly  visible,  impacts  of 
Evolutionary  Acquisition 

•  Need  to  understand  impacts  using  many  strategies 


Built  computer  simulation  model  of  work  and 
information  in  an  Evolutionary  Acquisition  project 


,  .  ^  ^  Naval  Postgraduate  School 

Acquisition  Research  Program:  Creating  Synergy  for  Informed 


Information  Flows  in  a  Single-block 
Acquisition  Project 

Time  Periods 

1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21 


Develop  Requirements 


Develop  Technologies 


Advanced  Developmen 


Manufacturing 


User  Product  Testing 


Milestones 


Models  inter-phase  concurrence  &  information 

dependencies 
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Work  Flows  and  Backlogs  through  a 

Development  Phase 


Information  Flows  in  an  Incremental 

Acquisition  Project 


Time  Periods 


Develop  Requirements 


Develop  Technologies 


Advanced  Developmen : 


Manufacturing 


User  Product  Testing 

Milestones,  Iter  #1 
Milestones,  Iter  #2 
Milestones,  Iter  #3 


8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30 


•  Contracting,  etc.  modeled  with  indirect  work  at  start  of  each  phase 

•  Reviews  modeled  with  indirect  work  at  end  of  each  phase 
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Modeling  Performance  and  Resources 


*  Acquisition  Project  Performance 

-  Schedule  -  when  how  many  requirements  are  satisfied 

-  Totai  project  cost  (iabor  as  proxy) 

-  Risk  of  satisfying  requirements  by  a  deadiine 

*  Resources 

-  Two  workforces'.  Deveiopment  &  Project  Management 

-  Resource  progress  rate  =  ailocated  workforce  *  productivity 

-  Deveiopment  aiiocated  to  reduce  work  backlogs 

-  Project  management  aiiocated  to  coordinate  development  activities 


Acquisition  Research  Program:  Creating  Synergy  for  informed  Schoc|l 


17 


Model  Calibration  and  Testing 


•  Model  was  calibrated  to  Javelin  project 


Active  Phases 


Work  packages 
being  deveioped 

(also  proxy  for 
development  effort) 


Work  started  and  active  PhIt[Requirements,Iterl]  :  JavelinCalibration  'i — 
Work  started  and  active  PhIt[Techno logy, Iter  1]  :  JavelinCalibration  — 2— 

Work  started  and  active  PhIt[Design,Iterl]  :  JavelinCalibration  — 3 - ^ 

Work  started  and  active  PhIt[Manufacturing,Iterl]  :  JavelinCalibration  — 
Work  started  and  active  PhIt[Use,Iterl]  :  JavelinCalibration  — 0  0 


—  work  packages 
■«-  work  packages 

work  packages 

—  work  packages 

—  work  packages 


Model  structure  and  behavior  is  consistent  with  the  Javelin  project 


Acquisition  Research  Program:  Creating  Synergy  for  informed  Cha^^^g^y®  School 


Impacts  of  Multiple  Development  Blocks 


Requirements 
Tested  and 
Approved  by  Users 

(%  of  all  project 
requirements) 
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Impacts  of  Multiple  Development  Blocks 


Units  of 
Measure 

Pn 

Javelin 
(single  block) 

oiect  Scenar 

Base  Case 
(single  block) 

io 

Base  Case  (3 
blocks) 

Best 

Performance 

Duration  to  first 
requirement  satisfied 

weeks 

471 

470 

397 

Base  Case 
(3  blocks) 

^  Duration  to  max. 

^  requirements  satisfied 

weeks 

520 

518 

762 

Base  Case 
(single  block) 

0) 

^  Total  deveiopment  cost 
(0 

$1,000,000 

722 

719 

1,555 

Base  Case 
(single  block) 

E 

O  Requirements  satisfied 
by  deadiine 

0) 

%of 

requirements 

developed 

100 

91 

18 

Javelin  (single 
block) 

Finai  requirements 
satisfied 

%of 

requirements 

developed 

100 

91 

91 

Javelin  (single 
block) 

The  (dis)advantages  of  Evolutionary  Acquisition  depend 
on  what  performance  measures  are  most  important. 
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Managing  Iterative  Development 


Base  Case 
(3  blocks) 
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Managing  Iterative  Development 


25 


Number  of 
Active 
Phase 
Interfaces 


20 


Base  Case 
(3  blocks)  ^ 


Base  Case 
(single  block) 


i 


0  45  90  135  180  225  270  315  360  405  450  495  540  585  630  675  720  765  810  855  900 

Time  (Week) 

Active  phase  and  bloek  interfaees  :  BaseCase-SingleBloek 
Aetive  phase  and  bloek  interfaees  :  BaseCase-Spiral  - 2 


^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^  dimensionless 
— 2 — 2 — 2 — 2 — 2 — 2 — 2 — 2 — 2 — 2 — 2 — 2 — 2 — 2 — 2 — 2 — 2 —  dimensionless 
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Managing  Iterative  Development 


Requirements 
Tested  and 
Approved  by 
Users 


125 


100 


75 


50 


25 


Base  Case  with  more 
developers  and  more 
project  management  -  / 

$1 ,753m!! 


1<-  Base  Case 
(3  blocks)  -  $939nr 

Base  Case  with  more^ 
developers  -  $1 ,761  m 


0  45  90  135  180  225  270  315  360  405  450  495  540  585  630  675  720  765  810  855  900 

Time  (Week) 


Average  Pereent  of  Requirement  Provided  :  JavelinCalibration  — ^ ^ ^ ^ ^ ^ —  Dmnl 

Average  Pereent  of  Requirement  Provided  :  JavelinSEvenBloeks  —2 - 2 - 2 - 2 - 2 - 2-  Dmnl 

Average  Pereent  of  Requirement  Provided  :  JavelinSEvenBloeksMoreDev  — ^ ^ ^ - s  Dmnl 

Average  Pereent  of  Requirement  Provided  :  JavelinSEvenBloeksMoreDevPM  4  4  4  Dmnl 
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Conclusions  -Evolutionary  vs.  Single  Block 
Development  Approaches... 


•  First  Unit  Equipped  with  some  (but  not  all)  requirements 
satisfied  faster 


•  Satisfies  requirements  in  multiple  steps 

•  Requires  more  time  to  satisfy  all  requirements 


•  Costs  more  than  single-block  development  for  same 
requirements 

•  High  risk  of  not  satisfying  all  requirements  by  the  time 
single-block  development  can  satisfy  all  requirements 
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Implications  for  Evolutionary  Acquisition 

Project  Managers 


•  More  development  phases  and  activities  to  manage  and 
coordinate:  larger  and  different  PM  needs 

•  More  concurrence  and  resulting  complexity:  bottlenecks 
change  and  move... are  more  difficult  to  identify  and 
manage  ^  focus  more  on  this 

•  Creates  counterintuitive  behavior  (e.g. reductions  in  project 
cost  by  adding  resources)  -  opportunities  to  improve 
performance... IF  you  develop  a  deep  understanding  of 
the  drivers  and  constraints  of  Evolutionary  Acquisition 
progress. 

Need  more  investigation  of  more  EA  projects. 


Our  Bottom  Line  on  Risks 


•  DoD  uniquely  outsources  development  for  internal  use 

-  owns  the  product  over  its  life  cycle 

•  There  are  inherent  potential  risks  with  incremental  development 

-  inefficiencies  from  re-work  (duplication) 

-  risk  of  project  error  (from  discontinuous  membership) 

-  organizational  impacts  (queuing  theory) 

-  relative  concurrency  drives  risk 

-  variety  in  the  fleet  (support,  failure  cause,  training,  etc.) 

•  Don’t  defer  what  you  can  do  now 

•  Defer  what  you  cannot  do  now  -  tech  readiness 

•  Product  attributes  may  affect  development  strategy 
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Our  Top  Line  on  Control 


•  Rigorous  Preliminary  Effort  on  Architecture 

•  Meticuious  Configuration  Management 

•  Individuai  Accountabiiitv 

•  Other  control  measures  to  balance  risks 


$$ 


Total  Cost 


Control 


-  T&E,  Interface  Control,  Peer  Review 

-  GPR,  MOSA  &  OA  Incentives,  etc. 

“inteiligent  design  is  way  faster  than  evolution.” 

Robert  N.  Metcalfe 


Risk 


Perceived  Relationships  Among  Project 
Cost,  Control  and  Risk 
(adapted  from  Wysocki  2003) 
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Questions? 


